Multi-input Fusion Spelling Error Correction Model Based on Contrast Optimization
WU Yaoyao1,2,3, HUANG Ruizhang1,2,3, BAI Ruina1,2,3, CAO Junhang1,2,3, ZHAO Jianhui1,2,3
1. Engineering Research Center of Text Computing and Cognitive Intelligence of the Ministry of Education, Guizhou University, Guiyang 550025; 2. State Key Laboratory of Public Big Data, Guizhou University, Guiyang 550025; 3. College of Computer Science and Technology, Guizhou University, Guiyang 550025
Abstract：Chinese spelling correction is essential in text editing. Most of the existing Chinese spelling error correction models are single input models, and there are limitations in the semantic information and error correction results of the models. In this paper, a multi-input fusion spelling error correction method based on contrast optimization, MIF-SECCO, is proposed. MIF-SECCO contains two stages: multi-input semantic learning and contrast learning-driven semantic fusion error correction. In the first stage, preliminary error correction results from multiple single input models are integrated to provide sufficient complementary semantic information for semantic fusion. In the second stage, multiple complementary sentence semantics are optimized based on the contrastive learning approach to avoid over-correction of sentences by the model. The limitations of error correction results of the model are improved by fusing multiple complementary semantics for re-correction of erroneous sentences. Experimental results on the public datasets SIGHAN13, SIGHAN14 and SIGHAN15 demonstrate MIF-SECCO effectively improves the error correction performance of the model.
[1] DEVLIN J, CHANG M W, LEE K, et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding // Proc of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies(Long and Short Papers).Stroudsburg, USA: ACL, 2019: 4171-4186. [2] BROWN T B, MANN B, RYDER N, et al.Language Models Are Few-Shot Learners // Proc of the 34th International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2020: 1877-1901. [3] YANG Z L, DAI Z, YANG Y M, et al.XLNet: Generalized Autoregressive Pretraining for Language Understanding // Proc of the 33rd International Conference on Neural Information Processing Systems. Cambridge, USA:MIT Press, 2019: 5753-5763. [4] CHANG T H, CHEN H C, YANG C H.Introduction to a Proofrea-ding Tool for Chinese Spelling Check Task of SIGHAN-8 // Proc of the 8th SIGHAN Workshop on Chinese Language Processing. Stroudsburg, USA: ACL, 2015: 50-55. [5] CHU W C, LIN C J.NTOU Chinese Spelling Check System in SI-GHAN-8 Bake-off // Proc of the 8th SIGHAN Workshop on Chinese Language Processing. Stroudsburg, USA: ACL, 2015: 137-143. [6] WANG Y R, LIAO Y F.Word Vector/Conditional Random Field-Based Chinese Spelling Error Detection for SIGHAN-2015 Evalu-ation // Proc of the 8th SIGHAN Workshop on Chinese Language Processing. Stroudsburg, USA: ACL, 2015: 46-49. [7] ZHANG S Y, XIONG J H, HOU J P, et al. HANSpeller++: A Unified Framework for Chinese Spelling Correction // Proc of the 8th SIGHAN Workshop on Chinese Language Processing. Stroudsburg, USA: ACL, 2015: 38-45. [8] WANG D M, SONG Y, LI J, et al. A Hybrid Approach to Automa-tic Corpus Generation for Chinese Spelling Check // Proc of the Conference on Empirical Methods in Natural Language Processing. Stroudsburg, USA: ACL, 2018: 2517-2527. [9] ZHANG S H, HUANG H R, LIU J C, et al. Spelling Error Correction with Soft-Masked BERT // Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2020: 882-890. [10] WANG D M, TAY Y, ZHONG L.Confusionset-Guided Pointer Net-works for Chinese Spelling Check // Proc of the 57th Annual Mee-ting of the Association for Computational Linguistics. Stroudsburg,USA: ACL, 2019: 5780-5785. [11] CHENG X Y, XU W D, CHEN K L, et al. SpellGCN: Incorporating Phonological and Visual Similarities into Language Models for Chinese Spelling Check // Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2020: 871-881. [12] HUANG L, LI J J, JIANG W W, et al. PHMOSpell: Phonological and Morphological Knowledge Guided Chinese Spelling Check // Proc of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Long Papers). Stroudsburg, USA: ACL, 2021: 5958-5967. [13] WANG B X, CHE W X, WU D Y, et al. Dynamic Connected Networks for Chinese Spelling Check // Proc of the Findings of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2021: 2437-2446. [14] XU H D, LI Z, ZHOU Q Y, et al. Read, Listen, and See: Leve-raging Multimodal Information Helps Chinese Spell Checking // Proc of the Findings of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2021: 716-728. [15] LI Y H, MA S R, ZHOU Q Y, et al. Learning from the Dictionary: Heterogeneous Knowledge Guided Fine-Tuning for Chinese Spell Checking // Proc of the Findings of the Association for Computational Linguistics. Stroudsburg,USA: ACL, 2022: 238-249. [16] LI C, ZHANG C Y, ZHENG X Q, et al. Exploration and Exploitation: Two Ways to Improve Chinese Spelling Correction Models // Proc of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Short Papers). Stroudsburg, USA: ACL, 2021: 441-446. [17] LI Y H, ZHOU Q Y, LI Y N, et al. The Past Mistake Is the Future Wisdom: Error-Driven Contrastive Probability Optimization for Chinese Spell Checking // Proc of the Findings of the Association for Computational Linguistics. Stroudsburg, USA: ACL, 2022: 3202-3213. [18] VASWANI A, SHAZEER N, PARMAR N, et al.Attention Is All You Need // Proc of the 31st International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2017: 6000-6010. [19] SUN Z Q, LI Z H, WANG H Q, et al.Fast Structured Decoding for Sequence Models // Proc of the 33rd International Conference on Neural Information Processing Systems. Cambridge, USA: MIT Press, 2019: 3016-3026. [20] WU S H, LIU C L, LEE L H.Chinese Spelling Check Evaluation at SIGHAN Bake-Off 2013 // Proc of the 7th SIGHAN Workshop on Chinese Language Processing. Stroudsburg, USA: ACL, 2013: 35-42. [21] YU L C, LEE L H, TSENG Y H, et al. Overview of SIGHAN 2014 Bake-off for Chinese Spelling Check // Proc of the 3rd CIPS-SIGHAN Joint Conference on Chinese Language Processing. Strou-dsburg, USA: ACL, 2014: 126-132. [22] TSENG Y H, LEE L H, CHANG L P, et al. Introduction to SIGHAN 2015 Bake-off for Chinese Spelling Check // Proc of the 8th SIGHAN Workshop on Chinese Language Processing. Stroudsburg, USA: ACL, 2015: 32-37. [23] LI J H, WANG Q, MAO Z D, et al. Improving Chinese Spelling Check by Character Pronunciation Prediction: The Effects of Adaptivity and Granularity // Proc of the Conference on Empirical Me-thods in Natural Language Processing. Stroudsburg, USA: ACL, 2022: 4275-4286.
2024, Vol. 37 
